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Abstract 

This research evaluates an English language textbook from CLIL perspective. For this purpose, 
an intermediate level (grade-11) English language textbook has been selected and analyzed 
utilizing Coyle’ s conceptual framework of 4Cs (i.e. content, cognition, communication and 
culture). Content, communication, and culture have been explored through a checklist, whereas 
cognition has been explored by developing a corpus from the questions given in the exercises 
of the textbook and analyzing in the light of Bloom’ s taxonomy. The results reveal certain 
breeches between CLIL features and the textbook’ s contents. Layout, learning outcomes, 
organization of the content, subject matter, authenticity of the text, exercises, and focus 
on language skills does not seem to match with CLIL perspectives. Listening and speaking 
skills are observed to be ignored. Moreover, the exercises do not seem to foster critical 
thinking and interaction between students and teachers. Most of the questions are observed 
covering only first two levels (i. e. knowledge and comprehension) of Bloom’ s taxonomy. The 
study concludes that CLIL principles are not integrated in the textbook. Therefore, the 
textbook is not suitable to an ESL/EFL setting. 

Keywords: Bloom’ s taxonomy; content and language integrated learning; textbook evaluation 

1. Introduction 

CLIL has been described as the most recent developmental stage of the 
communicative language teaching (Dalton-Puffer, 2007; Georgiou, 2012). CLIL is an 
acronym that is used as a generic term that has been given numerous interpretations. 
Marsh (1994 in Ioannou-Georgiou & Pavlou, 2011) defined this acronym initially as 
“Content language integrated learning” that refers to such situations in which 
subjects or parts of subjects are taught in a foreign language with dual focuses i.e. 
content and a foreign language learning simultaneously. Coyle (1999) stated that CLIL 
is an influential pedagogical tool aiming to safeguard the particular subject being 
taught while promoting another language other than mother tongue for learning purpose 
as well as with an aim of learning process itself. The main idea behind this approach 
is to teach the students the content of a subject and at the same time, learn a 
foreign language as a learning tool. Due to this reason, content language integrated 
learning cannot be defined as a methodology of language learning. The first 
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approximation, towards this concept, is to consider content language integrated 
learning as a convergence of many ELT methods and not mere as unique and isolated 
teaching trend. CLIL contains a number of approaches and methods that could be seen 
on a continuum, at one end, greater interest in teaching instruction through L2 and 
at the other end, focus on second or foreign language learning (Banegas, 2011). 
Lasagabaster (2008) states that CLIL is the integration of content with language or 
learning with the help of language. The learning can be optimized by providing the 
learners with an opportunity to context-based education that results in a dialogue to 
accomplish language teaching objectives (Hadley, Long & Luna, 2000). Similarly, Mohan 
(1986) declared that language teaching is inadequate without incorporating subject 
matter and immersion method is useful to teach second language (Mangubhai, 2005). 

Since teaching a language through specific subjects has not been identified as 
effective, rather CLIL may also present to acquire a foreign language to attain more 
successful subjects. CLIL is mistakenly considered as a language teaching method, as 
it fosters other essential skills for child development. Cultural awareness and 
cognitive ability are the cases of it. In fact, CLIL is socially connected to 
bilingualism. Therefore, CLIL model has its roots in bilingual education. 

Integration lies in the heart of CLIL teaching approach. The framework of 4Cs 
(see Figure 1), developed by Coyle (2005), offers essential principles to content 
language integrated learning approach. These principles provide guidance for 
selecting CLIL-focused textbooks. He defined the 4C as four dimensions i. e. (1) 
content: it provides opportunities of rich context that turns language acquisition 
into practices; (2) communication: it enables the learners to improve overall 
competence in target language. It helps develop oral communication and deepen 
learners’ awareness in both i. e. target and mother languages. It assists in 
developing plurilingual interests and attitudes. Both writing and speaking are 
emphasized. In this way, students use language to learn and learn to use language 
cognition and culture; (3) cognition: in teaching-learning process, cognition lies 
in the heart of the model and it refers to thinking skills that should be developed 
in order to link concrete and abstract learning, formation of the concepts and 
understanding to the language; (4) culture: activities and lesson provide learners 
with shared understandings and alternative perspectives which deepen understanding 
among learners about self and otherness. Consequently, CLIL builds intercultural 
understanding and knowledge. In addition, it helps introduce wider cultural context 
to learners and develop their intercultural communication. 
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The 4Cs conceptual framework for CLIL 



Figure 1: Conceptual Framework for CLIL 

A crucial aspect is to rethink about language using and language learning in 
CLIL, according to Coyle (2005) a foreign language cannot be packed into grammatical 
developments leaving complex linguistic construction and past tenses “until later” . 
Coyle reflects on one of the main differences between language using and language 
learning. In CLIL context, language use is different from the language use in 
language classrooms i. e. the students need to argue, discuss, justify, explain and 
debate, and issues linked to language content use and adequate content for specific 
vocabulary studies. This is one of the main reasons why scaffolding a language is 
important in CLIL classrooms. The taught component of content determines the language 
for CLIL classrooms, for this reason, defining the language for learning, through 
learning and for learning, aka The Language Triptych (see Figure 2), becomes one of 
the crucial issues in CLIL approach (Coyle, Hood & Marsh, 2010). Language of learning 
refers to the learners and language that learners are required to access basic skills 
and concepts related to the content. Language for learning particularly focuses on 
language to enable individuals to learn in foreign language settings i. e. how to 
develop learning strategies, how to operate group discussion, how to hypothesize and 
summarize and ask questions that challenge cognitively. Language through learning 
refers to the notion that language learning cannot take place without active 
participation in thinking and language. CLIL classroom settings demands different 
level of interaction and talking as compared to traditional language classroom. 
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Figure 2: Language Triptych 


CLIL is recent approach to develop material and evaluate textbooks in an area 
which is still to be developed. This research adopts tentative CLIL textbook 
evaluation checklist. From last three decades, numerous checklists have been 
developed to evaluate ELT textbooks but this research focuses on the elaboration of 
checklist that should have main characteristics of CLIL. Textbook evaluation 
checklist consists of a set criterion. Evaluation checklists are useful instruments 
to select, adapt or evaluate textbooks for teachers (McGrath, 2002; Sheldon, 1988). 
McGrath (2002) points out that it is not a watertight category. It differentiates 
system (objective) with impression (subjective), cost effective, significant 
information can be obtained a within short time period. Moreover, it provides easy 
format to informants to provide information. 

Checklist limitations depend on inclusion criteria that are bound with context 
during its creation and according to the need to adapt some checklist and tailor the 
checklist in terms of specific context. Demir and Ertas (2014) summarize the fact 
that in the light of literature no checklist is complete. In spite of the item number 
it is made up of, it can modify by deleting or adding items depending on the 
instructional settings. Preeminent theorists in ELT textbook design and evaluation 
field like Brown (1995), Cunningsworth (1995), Sheldon (1988) and Williams (1983) 
mutually agreed that textbook checklist evaluation criteria should have pertaining 
criteria to physical characteristics such as layout, organization and other 
logistical characteristics. Other criteria should also be taken into consideration 
like assess textbook methodology, approaches and aims and define the degree to which 
extent material is not only teachable but fits according to the needs of learners and 
teacher and overall organization of the curriculum, language skills, vocabulary, 
grammar as well as linguistic items to the dominant socio~cultural environment, 
gender, culture and to determine the extent i. e. subject, culture, linguistic item 
and topics match with the personality of the students, needs, interests, backgrounds 
of the students and institutes. 
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ELT practitioners help evaluate the efficacy of language teaching materials 
from different perspectives through evaluation checklist e. g. (1) predictive and 
retrospective (Ellis, 1997), (2) pre-use, in-use, and post-use (Tomlinson, 2003). 

Littlejohn (1998, 2011) suggested three levels for evaluation (i) What is there? (ii) 
What is required of users? (iii) What is implied? Evaluation checklists, prepared by 
leading experts, provide criteria for textbook evaluation in detail. McGrath (2002) 
proposed two-stages criteria i. e. (a) first glance (b) in-depth. The other well- 
established checklists have been proposed by Tucker (1975), Daoud and Celce-Murcia 
(1979), Sheldon (1988), Cunningsworth (1995), Ellis (1997), McGrath (2002), Mukundan 
and Ahour, (2010). Skierso (1991) proposed an evaluation checklist in which the 
material had been divided into five sections: bibliographical data, aims and goals, 
lay out, subject matter, vocabulary and structure and physical make up. Garinger 
(2001 in AbdelWahab, 2013) stated that while evaluating a textbook three contents are 
required to be evaluated i.e., depth and breadth of material, teaching objectives and 
whether the textbook is needed to be supplementary material or not. This evaluation 
checklist was consistent with the evaluation checklist proposed by Cunningsworth 
(1995) i. e. one of the most considerable work in ESL/EFL textbook evaluation. His 
checklist included 45 criteria and eight categories for the general evaluation of the 
textbook i. e. aims and approaches, organization/design, study skills, language 
content, topic, methodology, teacher’ s manual and practical consideration. 

Stardling (2001) analytical framework constructed the framework of four 
categories and posed forty probing questions. The first category dealt with 
evaluation of coverage, textbook content, sequencing and curriculum, incorporation of 
multi-perspectives, regional and cultural categories and omissions. Second category, 
identifying pedagogical value of the textbook, raises questions related to students’ 
prior knowledge, whether the textbook promotes skill-based learning or memorization, 
on use of pictures or charts, on explanation of historical concepts, and on fostering 
critical thinking. The third category, identifying intrinsic qualities of the history 
textbook, included questions on textbook pitch, on whether the textbook depends on 

reductionism, identifying authors’ bias. The last category consisted on extrinsic 

factors that may impact on the book. A question to discover when the book was first 
available in the market, robustness and price of the book, whether the book is 

designed for specific group of students and to which extent the textbook required 
alternative resources are included in this category. Stardling (2001) evaluation 
checklist question can be refined, re-clustered and it requires more categories. He 
proposes different categories as compared to Stardling checklist like 

‘readability’ , ‘format’ , and ‘quality of the text’ but these questions are 

quite similar to Stardling checklist. Stardling referred to analytical criteria for 
material evaluation to be used by selection committees of textbooks and teachers. 

Most of the researches have been conducted on textbook evaluation by applying 
ELT based evaluation checklists ignoring specific elements like content contextual, 
communicative, cognitive aspects. For instance, Jahangard (2007) developed an 

evaluation checklist to evaluate four EFL textbooks prescribed by Iranian high school 
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by Ministry of Education. The merits and demerits of the textbooks were discussed 
with reference to 13 common extracted features from different checklists. The 
evaluation criteria included explicit objectives, vocabulary explanation, visuals, 
educational approaches, topic and task, review and test section, layout, authentic 
language, graded content, clear instructions, presentation and practice of grammar, 
learning strategies and fluency in four language skills. The evaluation checklist was 
evaluated by teachers. Results showed that the textbook focused only reading skills 
and grammar practice ignoring up-dated topics, incompatibility between word meaning 
in the ‘New Word Section’ and ‘Reading Comprehension Section’ . The study was 
subjective and theoretical. It also involved author’ s opinion without user’ s 
feedback. 

Litz (2005) evaluated a textbook used for undergraduate English students of 
South Korea through a questionnaire filled in by teachers and students. His 
questionnaire was designed to evaluate overall suitability and overall pedagogical 
value of the language program of the textbook including layout and design, practical 
consideration (price, methodology, accessories etc.), skills integration and 
appropriateness, balance and range of activities, cultural and social considerations, 
and language types and subject content in the textbook. The findings showed that 
textbook was communicative and focused on multi-skills. Activities involved both 
communicative and controlled practice of language skills. Students were allowed to 
give freer response. The researcher found shortcomings in the textbook through 
teacher and students filled checklist e. g. repetition of activities, lack of 
realistic discourse, lack of encouragement of meaningful practice. The use of 
questionnaire was not persuasive enough as questionnaire had its own weakness. 

Dat (2008) evaluated nine ELT textbooks by interviewing teachers, 
administrators and policy makers in South Asian context. He used questionnaires also 
to describe the perceptions of teachers and students. The findings identified the 
positive features e. g. national identity and cultural knowledge, knowledge of 
regional events, awareness about global integrity, analytical thinking and SL as a 
pedagogical tool. Its drawback was the uncommunicative use of English, unauthentic 
use of English language, no writing practice, and poor effective engagement. The 
textbook lacking to meet the principles of content language integrated learning. 

In the context of Pakistan, especially with reference to Punjab province, few 
researches have been conducted on textbook evaluation. Naseem, Shah and Tabassum 
(2015) evaluated English textbook approved by Punjab Textbook Board (PTB) for 9 th 
class. The data was collected through an evaluation checklist which was designed by 
reviewing well-developed checklists. The checklist comprised of different levels i.e. 
(1) First Glance (different activities of the textbook), (2) Close Evaluation 
(Physical and utilitarian attributes, (2) Efficient outlay of objectives and 
supplementary materials, and (4) learning-teaching content. The findings showed that 
PTB English textbook for 9 th class was incompatible with the needs of promoting 
integrated skills and also failed to meet general objectives. There was no balance 
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between language skills. Most importantly, the textbook did not cater needs of 
variety of learners. 

Kausar, Mushtaq and Badshah (2016) conducted a study to evaluate an English 
textbook from both teachers’ and students’ perspective. The questionnaire was 
adapted from a checklist by Litz (2005) covering six perspectives i. e. outline and 
planning, language skills, exercises, type of language, overall view, topic and 
theme. The data was collected from 100 students and 10 teachers. The findings 
revealed that there was shortfall in the organization, subject matter, outline and 
planning as well as exercises. The teachers and learners did not recommend this book 
for future. The textbook needed revisions as it did not cater students’ needs. 
Nazeer, Shah, Sarwat (2015) adapted the evaluation checklists proposed and revised by 
(Mukundan, Hajimohammadi & Nimehchisalem, 2011) in their study. The results 
highlighted a number of short comings e. g. absence of visual aspects. The researchers 
recommended revising the textbook to include language skills. Ahmad, Ismail and Saba 
(2019) evaluated the content of English-2 approved by Punjab Curriculum and Textbook 
Board (PCTB), Lahore, Pakistan applying a checklist based on communicative language 
teaching principles. The results revealed breeches between the textbook’ s content 
and communicative language teaching principles. Therefore, revisions were recommended 
in the content of the said textbook. Thus, it can be assumed that Pakistani textbooks 
are not suitable to meet the communicative needs of the learners. 

According to the claim of Georgiou (2012) CLIL is the most recent 
developmental stage of the communicative language teaching (CLT). A review of the 
past researches shows that there is no such research as has evaluated English 
textbooks from communicative language teaching perspective and content analysis. 
Content analysis checklists/questionnaires are used to analyze the content of the 
English language textbooks. The questionnaires were designed to take teachers’ and 
students’ perspectives about the content of the book that might be subjective. There 
is lack of proper research held to delve deep in English textbooks from CLIL model in 
Pakistani context. However, some studies have been conducted in other countries based 
on CLT principles’ evaluation checklists e. g. Liao (2000) conducted a research in 
China to investigate the application of communicative language teaching principles. 
His results showed that three difficulties were hurdles in the way of successful 
application of communicative teaching method i. e. lack of target language cultural 
knowledge, no familiarity with new emerging communicative method, negative influence 
of old educational tradition on teacher. Furthermore, language and culture are 
inseparable so that Chinese are not aware of target language due to this they are not 
proficient in English language and lack of authentic material is another key factor. 

Mustafa (2009 in Noori, 2018) carried out research in Indonesia through 
classroom observation. He identified real position of CLT in Indonesian educational 
settings. He argued that communicative language teaching was not coping with the 
needs of the students. He added that practicing communicative language teaching was 
not easy in Indonesian educational context. The key factors of failure of 
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communicative language teaching were time constraint, teacher’ s less proficiency in 
English language, emphasis on written exams, absence of authentic material, 
teachers’ emphasis on form rather content, absence of real-life communication and 
social interaction that helped the students perform outside the classrooms. 

Aftab (2012) evaluated a seven series of ELT textbooks used in Punjab in 
public and private sector institutes. Data was collected through questionnaires, 
interviews and documentation including participants, officials, school 
administrators, English language teachers and students. CLT textbook evaluation 
checklist was also employed. The author concluded that textbooks like ‘Every Day 
English Book 6’ orders, instructions, and polite requests in: Guided English Book 
0” were linguistically appropriate and focused on functions of language. Some books: 
English Book 6; Every Day English Book 6; Oxford Progressive English Book 5; Oxford 
Progressive English Book 6, have useful topics and integrated skills learning. Few 
books: Oxford Progressive English Book 5, Oxford Progressive English Book 6, had for 
vocabulary building and variety of authentic reading text. He concluded that writing 
and listening skills were ignored and speaking was unrealistic (speaking skills also 
focused on ‘Guided English 0’ ). The content suggested for oral practice in “Oxford 
Progressive English Book 6” were either selected from text or given subjectively. The 
textbooks were focused on accuracy rather than communicative potential. However, he 
did not investigate the organization and activities of the textbooks in detail. 

The literature on ELT material development is extensively available and easily 
accessible. However, the area of textbook evaluation is under-researched, with 
limited relevant literature (Dendrinos, 1992; Litz, 2005; Sheldon, 1988). The 
researches on textbook evaluation mostly used general ELT evaluation checklists. Up 
till now, a few researches in Pakistan have been conducted to evaluate English 
language textbooks from ELT perspectives including predictive and retrospective 
methods but a few components of CLIL checklist have been incorporated in them. In 
addition, there is no such research as investigated 4Cs of CLIL model in English 
language textbooks. Therefore, this study aims to investigate the content of an 
intermediate level English language textbook through a CLIL based checklist in order 
to find out the integration of language and content at four dimensions i. e. content, 
cognition, culture and communication. This study is pioneering in its nature and 
focuses to answer the following questions: 

1. How well Intermediate English textbook is structurally and physically 
organized? 

2. How do the activities of the textbook integrate content and language and 
covers needs of ESL learners for language learning? 

3. Do activities promote student-students and teacher-student interaction to 
achieve communicative competence and foster critical thinking? 

4. Does the English textbook promote intercultural or biased views of culture 
among ESL learners? 
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2. Methodology 

This study is qualitative and quantitative in nature. The researchers created 
a tentative content language integrated learning textbook evaluation checklist taking 
into consideration two areas i. e. by reviewing the existing instruments for ELT 
textbook evaluation and established principles for CLIL material production. It is 
important to mention that the textbook researchers have introduced a number of 
checklists (see Table 1) for evaluation purpose in different contexts and from 
different perspectives. 

Table 1: Checklists Introduced by Past Researchers 


Decade 

Year 

Checklist 

1970s 

1975 

Tucker 

1979 

Daoud and Celce-Murcia 

1980s 

1983 

Williams 

1988 

Sheldon 

1990s 

1991 

Skierso 

1995 

Cunningsworth 

1996 

Ur 

1998 

Littlejohn 


Munkundan and Ahour (2010) reviewed the textbook evaluation checklists, 
(introduced from 1970 to 2000), and claimed that the textbook evaluation checklists 
were either qualitative (see e. g. Cunningsworth, 1984, 1995; Dougill, 1987; 

Griffiths, 1995; Hemsley, 1997; Hutchinson & Waters, 1987; Karamoozian & Riazi, 2008; 
McGrath, 2002; Sheldon, 1988) or quantitative (see e. g. Daoud & Celce-Murcia, 1979; 
Litz, 2005; Miekley, 2005; Sheldon, 1988; Skierso, 1991; Tucker, 1975; Williams, 
1983) or outline format/head words i. e. without questions and rating scales (see e.g. 
Ansari & Babaii, 2002; Brown, 1995; Littlejohn, 1998). Their reviews, for the 
evaluation instruments to identify a well-proposed evaluation checklist, eventually 
helped Munkundan, Hajimohammadi and Nimechisalem (2011) develop a better checklist to 
evaluate ELT materials. 

In the field of textbook evaluation, methodology is rarely described in-depth. 
Research procedures, data processing and interpretation are taken into account while 
writing methodology. The methods for textbook evaluation can be classified into 
detailed categories i. e. (1) methods of theoretical analysis (i) the special- 
analytical method (ii) the theoretical-analytical methods (iii) the comparative 
analysis of textbook (2) empirical analytical methods i.e. (i) investigate 
experimentally in textbooks use (ii) public inquiry applied to teachers (iii) Public 
inquiry applied to learners and (3) statistical methods (Hrehovcik, 2002). 

Literature shows that the methods of textbook evaluation are comprised of 
three ways i. e. impressionistic method, checklist method and in-depth method. This 
study has employed all of these three methods in order to investigate content 
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language integrated learning in an English language textbook. The CLIL evaluation 
checklist is adopted from Medina (2016). This checklist is designed by considering 
the common features of ELT and CLIL textbooks (e.g. language authenticity, 
availability and others). This is included in first section and other sections have 
included those features that totally or mostly refer to the content taught through 
second language. The checklist developer considered Mehisto’ s (2012) criteria for 
CLIL material production. According to Mehisto (2012) the quality of CLIL materials 
is highly multilayered and integrative and it helps increase the possibility that 
language learning and content will be meaningful. Other criteria e. g. visible 
learning process, fostering language skills’ development, or different types of 
formative assessment is included in Mehisto’ s list. 

The structure of the CLIL evaluation checklist is consisted on seven sections. 
Section is comprised of technical and general terms, sections 2 to 5 have the 
inclusion of 4Cs i.e. content, cognition, communication and culture. Textbooks based 
on CLIL, section 6 is related to language and last section 7 in the evaluation 
checklist is to reflect upon CLIL principles’ integration in the analysis of English 
textbook (Medina, 2016). 

The checklist is neither too broad nor too short to be filled in. While, 
analyzing the content, the textbook is used as a general term for methodologies in 
which text is codified under different categories (Ahuvia, 2001) i. e. (1) Item 
evaluation checklist method: this method is objective, systematic and easy to carry 
out in the way that items in the checklist are checked off in a certain order 
(AbdelWahab, 2013); (2) In-depth method: proposes a careful examination of grading 
and sequencing of representative features for instance the treatment of particular 
language skills, or design of a particular chapter. Cunningsworth (1995) stated that 
in-depth evaluation provides subsequently detailed evaluation of specific items in 
any textbook on areas like how exercises can cater for learners and syllabus needs. 
But this method lacks in overall evaluation of whole book. The researcher filled the 
checklist in yes or no responses by analyzing the items of checklist through in-depth 
content analysis of the textbook to avoid subjectivity of the teachers and students 
while taking their perceptions. 

Evaluation checklists, adapted to evaluate English textbooks according to CLIL 
specifications, are not numerous. The existing EFL/ESL evaluation checklists are not 
advisable to evaluate English textbooks under CLIL model. In CLIL, the main focus is 
on content and language revolving around 4Cs i. e. content, cognition, culture and 
communication. The previous checklists are ELT based. 

Moreover, Bloom’ s taxonomy (1956) has been employed to investigate the 
cognition level of questions in the textbook. For this purpose, a corpus was 
developed from the questions given in textbook’ s exercises. The purpose behind this 
step was to determine whether the activities fostered critical thinking among 
learners or not. The textbook, which is the subject of this study, is approved by 
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Punjab Curriculum and Textbook Board (PCTB), Lahore, Pakistan. This book is taught to 
the intermediate level (grade-11) students in Punjab (Pakistan). 

3. Observations and Discussions 

The evaluation of the textbook is carried by adopting checklist (see Checklist 
1) based on CLIL principles. The checklist has six sub-classes. The first sub-class 
consists of general having other sub-classes e.g. structure, supplementary material 
and physical utilitarian features, content, culture, communication, cognition and 
integration. 


Evaluation Criteria 

Yes 

No 

I. GENERAL 



A. Structure 



1. Is the organization consistent with the curriculum? 



2. Are the objectives specified specifically? 



3. Does it include self-evaluation at the modules? 



4. Does it provide summarizing items at the end of the modules? 



B. Supplementary material 



5. Is the resource pack complete? 



6. Does the resource pack include varied ICT resources? 



7. Does it offer supplementary material for under/over-achievers? 



8. Does it provide guidance for non-native content teachers? 



9. Does it provide support for language assistants? 



C. Physical and Utilitarian features 



10. Does it show quality in editing and publishing 



11. Is its layout attractive? 



12. Does it contain appropriate pictures, diagrams, tables? 



13. Is it easily available? 



14. Is it durable? 



15. Is it cost effective? 




3.1 General 

First sub-class comprised of the structure of the textbook, supplementary 
material for learners and teachers to facilitate learning process and physical 
characteristics of the textbook to attract the learners. 

3. 1.1 Structure 

The results show that the structure of the textbook does not match with the 
curriculum. The curriculum claims that the textbook fulfills the academic, job, 
social and functional needs of the learners. The structure of the textbook is neither 
satisfactory nor consistent with the structure of the book. There is no functional 
and logical organization between the content and subject of the book. The lessons are 
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difficult to comprehend by the learners because they are not systematically organized 
according to different themes. The lessons such as ‘Button Button’ , ‘Dark They 
Were Golden Eyes’ and ‘The Gift of Magi’ are organized by local bodies and written 
by natives because of that they are far away from the native culture. The topics are 
not seemed to be linked between units. Moreover, they do not coincide according to 
the stated function in the curriculum. The organization of the lessons can be 
considered as a bridge that fills the gap between claims made by the external for the 
materials and what will actually present inside the material themselves. 

The learning objectives are not clearly stated and no concise overview of 
topics, grammar/structures and language skills are given within each lesson that can 
be found in the table of the contents. Without specified objectives, objectives are 
not easily measurable as teacher are not trained in public sector schools according 
to the community’ s language teaching methods. Cunningsworth (1995) stated that pre¬ 
defined aims, objectives, and exercises should correspond to curriculum and it gives 
confidence to less experienced teachers and motivates learning but it is lacking in 
the textbook. Textbook’ s appropriateness to the aims and objectives of the 
curriculum (Cunningsworth, 1984) is considered as one of the guidelines that form the 
basis for formulating personalized checklist criteria directing at local relevance. 
So, the relevance of curriculum, aims and objectives should be related with the 
textbook (Aftab, 2012). There is no self-evaluation in the textbook. Richards and 
Rodgers (2014) declare that textbooks encompass learning objectives, language 
content, activities and roles of the instructor and students, learners’ self-study 
and provide assistance in second language learning (Saadipour & Shakouri, 2016). 
There are also no guides for the learners for self-study except the themes are given 
at the end of lessons that summarize the main idea of the lesson. There are no such 
questions as motivate the learners for self-study at home. Textbooks provide learners 
with efficient and additional collection of materials for their self-accessed 
learning and also for learners’ knowledge consolidation (Cunningsworth, 1995). In 
this way, textbooks save learners from teachers’ deficiencies and incompetency 
(Litz, 2005; 0’ Neil, 1982). The textbook is deficient in the organization of the 
textbook which is the most appealing thing to the stakeholders. 

3. 1. 2 Supplementary Material 

The textbook fails to provide any ICT package for teaching and learning. 
Audio-visuals aids are altogether ignored by the material developers. No teacher’ s 
guide/manual is available to aid the teacher as it is the main pillar of teaching. 
Nguyen, Nguyen and Then (2006) state that teacher’ s aid encourages learning 
motivation; develop learners’ creativity and learning styles; making optimal 
opportunities for language practice; combine all language skills; and use the 
textbook creatively. Teacher’ s aid should be given separately for time saving and 
conducive teaching-learning environment. It is a widely known fact that most general 
EFL courses provide teachers’ books as part of the whole materials package. They are 
important parts of the whole materials package because they have a considerable 
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influence on guiding the teachers. As Cunningsworth (1995) points out ‘a good 
teacher’ s book is invaluable in offering, among other things, guidelines on how to 
make the best use of the course’ (p. 112). No workbook is available for students for 
homework. The student workbook provides review exercises and a variety of practice 
exercises. It should be included to assist with the development of students’ 
proficiency with reading, grammar, spelling, writing, speaking vocabulary has 
enormous potential for classroom use or for homework assignments. No cassettes are 
available as supplementary material. Good quality cassettes with natural-sounding 
recordings of conversation, important points, pronunciation exercises, grammar and 
listening activities are helpful as teaching aid. No flash cards and posters are 
available with the textbook to facilitate teaching and learning. It does not provide 
any language assistance to non-native teachers. Only list of glossaries is given at 
the end of the lessons consisted on new vocabulary items extracted from the lesson in 
order to increase the mental lexicon the learners. Due to the inadequacy of 
supplementary material, learners and teachers use exam drilling supplementary 
material and guidebooks. 

3. 1. 3 Physical and Utilitarian Features 

The physical appearance of the English textbook has also shortcomings. The 
physical and utilitarian features include outside cover, binding, paper quality, 
printing, font size, space, adequate use of pictures. The content page list does not 
provide the details of knowledge areas including vocabulary, grammar, language 
skills. The content page introduces the uses and variety of English practice to 
learners. Moreover, it develops social skills, survival and functional skills 
(apologizing strategies, making requests etc.) in learners (Maris, Soars The findings 
reveal that textbook does not have positive impressionistic value to attract its 
stakeholders. The textbook is dull and outside is not informative at all (Naseem, 
Shah & Tabassum, 2015). It is hard enough to with-stand. Garvin (1988) has done lot 
of work on quality and quality product of the textbook and he has proposed eight 
dimensions on ‘product quality’ i.e., performance, features, reliability, 
conformance, durability, aesthetic, perceived and serviceability. 

In terms of physical state, ‘durability’ is relevant at this point of 
discussion. The quality of paper and its printing and binding should be durable for 
long term use. In aesthetics it includes: (a) title, font, binding and font size, 
page layout/format and illustrations; and (b) other graphics-acceptability with 
respect to level/age relevant to context and content (Khalid, 2009). The used font in 
the textbook is varied in the whole book. The font used in lesson content is 
appropriate and easily readable. On the other hand, the font used for stating themes 
of lessons, poems writing skills activities (sample/comprehension) is very small and 
white space is quite less which may distract reader while reading samples and 
attempting comprehension passages. 
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In PCTB’ s English textbook, exercises’ section is over-crowded due to less 
space. As the white space used for the content lesson is also congested. Usually, 
students write meanings of difficult words on the book. Due to less space, learners 
cannot write well. Every space must be used to the optimum, taking into consideration 
the space that enables readers to rest their eyes (Khalid, 2009). Ahymadpoor (2004) 
found the lack of high school students in English textbooks due to the boring, 
unattractive, incoherent texts and outdated, and incorrect, unattractive and 
inappropriate pictures of the book. There is no use of pictures, or diagrams etc. 
Textbook is accessible easily and is cost effective as well. So, the overall physical 
appearance and utilitarian features of the textbook are not attractive and durable. 
The textbook does not contain realistic or native content and it also lacks in day to 
day conversation that is the essential need of the ESL learners of Pakistan. The 
content of the textbook includes exercises and activities as well. 


3. 2 Content 


II. CONTENT 

Yes 

No 

16. Does it cover the content of the curriculum? 



17. Are learning outcomes for learning specified? 



18. Is the content appropriate for the students’ age? 



19. Is the content relevant to students’ experience? 



20. Is the order of sequence flexible? 



21. Does it provide support to simplify content (scaffolding)? 



22. Is the functional content functional? 



23. Are the activities suggested for practicing the content varied? 



24. Are the activities suggested enough for practicing the content? 



25. Is there authentic material at an appropriate level? 




By analyzing the content of intermediate English textbook, it is revealed that 
the content of the textbook are not updated and it does not match with the learners’ 
cultural background and their knowledge. As the content is written by foreigners due 
to that it is not according to the learners’ age and experience. The content is old- 
fashioned and not revised since long. It does not present modern and challenging 
authentic content to the learners. Liao (2000) noted that use of authentic 
instructional material designed by natives was too difficult for the foreign language 
learners with lower proficiency. He suggested using simple authentic material and 
realia. 


Furthermore, the content of the textbook does not contain rich learning 
content that can be used in practical use. The broader term content includes topics, 
functional content, exercises and activities that foster higher order skills, 
scaffolding, and authentic material. Many textbooks are scripted in an inappropriate 
and unnatural way for communicative purposes and do not prepare students adequately 
for pronunciation (Brazil, Coulthard & Johns, 1980; Levis, 1999) also structure, 
language, conversational rules, routines and strategies that they need to use in real 
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life (Yule, Matthis & Hopkins, 1992). Similar is the case in the present research 
findings. The textbook scarcely fulfills the required criteria of content which is 
required for language learning. Through the analysis of the Intermediate English 
textbook, tasks and exercises do not provide activities for learners to promote 
communicativeness and do not challenge them cognitively to think in target language. 
The learners are being controlled by the textbooks in solving questions. Kausar, 
Mushtaq and Badshah (2016) took the opinion of teachers and students, both were of 
the view that English textbook, for intermediate level learners; do not cater the 
needs of the learners. Moreover, the content of the exercises mainly laid emphasis on 
grammar like unit-based questions, correction of tenses and punctuation. Moreover, 
the textbook does not enhance critical thinking and creativity. Similar findings are 
observed in the research conducted in Turkey by Tok (2010). He highlighted that there 
was a major shortcoming in the English textbook due to the lack of creative 
activities. The learning outcomes are not specified in the textbook through which 
teacher can take guideline for achieving those objectives. The selection of the 
content is inappropriate in ESL context. 

During evaluation of the textbook, it is noticed that traditional learning 
processes (including language learning activities) are in effective. Due to complex 
and innovative content, textbook does not provide guidance to carry out activities 
and interaction in the class which ultimately fails to recognize modern teaching 
methodologies. There is no such activity and exercise as for practicing communication 
and listening skills while ignoring prosodic features. Most of the exercises are 
based on GTM that do not enhance their skills and motivate to learn further English. 
At this point, the effectiveness and utility of the textbook is questioned. Since, 
the curriculum does not have information in order to determine the fitness of the 
content that is laid out in the curriculum are according to learning objectives or 
not. Findings of a study by Shah, Majeed, ffaheed and Anjum (2013) showed that 
according to the learners’ and teachers’ response textbooks were just designed for 
academic learning rather than communicative needs. There is deficiency of such 
content as develop learners’ comprehension and critical skills, and enhance their 
productive ability. These two findings are aligned with the present research results 
that the major content of the textbook is consisted on exercises and it does not 
practice writing skills effectively. The questions, in exercises of intermediate 
English textbook, require answers in the form of composition comprising of 100-150 
words that solely focus on content of the unit or story and nothing to do with 
creativity (Kausar, Mushtaq & Badshah, 2016). So, the activities are enough to 
practice the content. The textbook does not have challenging, interesting and 
realistic topics. To some extent, it satisfied the need of diverse content of themes. 
However, content analysis of the textbook shows no authentic situation that is 
relevant to the Pakistani learners. In Pakistan, English language education is 
totally based on cramming to pass out the examination. There is no concept of real 
life in the textbooks (Shah et al., 2013). The PCTB’ s intermediate English textbook 
presents the content artificially and in a contrived way. 
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3.3 Cognition 


III. COGNITION 

Yes 

No 

26. Does it allow breaking down tasks/activities to make them more 
manageable (scaffolding) 



27. Are the activities cognitively appropriate for the content? 



28. Does it cater the needs of different learning styles? 



29. Do activities activate previous knowledge? 



30. Are activities challenging? 



31. Do activities include projects? 



32. Are activities motivating? 




The knowledge acquisition is connected intimately with comprehension. CLIL is 
incomplete without cognition dimension. Theorists like Baker (2011) conceive 
cognition as a fifth language skill after listening, speaking, reading and writing in 
EFL settings (Coyle, 2007). Cognition dimension deals with mental processes like 
remembering, repeating, questioning, solving problems and decision making. In an 
English textbook grammar and writing skills are mainly focused through practicing 
different activities. The activities in the textbook do not break down to make them 
more manageable. Tomlinson (2008) states textbook as humanistic one that values the 
learner as a human being and also helps learner in activities given in textbooks and 
in their learning process that connects mind with what is stated in the textbooks. 
Every individual has his own learning needs but the textbook does not cater learning 
styles of the learners. Numerous theorists like Long (1990) and Vygotsky (1978) 
advocated that cognitive value of learners’ social interaction or learner-learner 
interaction to promote learning. The given activities are not appropriate cognitively 
for the content. Thus, it measures low cognitive level of the students. Jacobs and 
Ball (1996) suggest that the most appropriate activities are those that encourage 
negotiation of the meaning and facilitate individual accountability through 
cooperative learning and promote positive interdependence. While analyzing the 
content, it is observed that activities do not activate prior knowledge of the 
learners through brain storming, pre-reading, or answer questions. The linguistic 
complexity in the text is challenging. Jolly and Bolitho (2011) proposed that the 
textbooks should incorporate activities which provide varied input of authentic 
language and require the learners to exploit the data using cognitive processes like 
analyzing (referred to as ‘language awareness activities’ ). Such tasks will make 
the students think critically ‘about how language is used for different purposes’ 
and provide real context to their struggles with language systems. 

Composition question answers are to some extent appropriate at this level. But 
through corpus analysis it has been determined that mostly questions are belonging to 
the category of Level-1 (i. e. knowledge) and Level-2 (i. e. comprehension) according 
to Bloom’ s taxonomy. There are total 115 short questions and 64 long questions. The 
short questions are all close-ended questions that evaluate the learners’ knowledge 
only. For instance, questions 3 (sub-questions) on page (9, 25, 30 etc. ) are all 
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knowledge questions that are challenging and not appropriate at point of level. The 
type of activity does not seek to enhance students’ critical thinking. In short, 
there is no cognitive progression. Similarly, there are 65 long questions to answer 
in 100~150 words which are also of comprehension level and the answer is from the 
content of the lesson. Mostly, questions have stem words (e.g. discuss, how, and why 
which are close-ended in nature), describe, compare and contrast) in order to check 
learners’ understanding to see whether they can summarize the whole idea or not. 
These questions develop the habit of cramming the content not language learning. It 
creates boredom in the class that is not reasonable pedagogically and will not enrich 
learners to answer independently. Thus, they cannot be effective for teaching¬ 
learning process. Activities should have an aim to promote cognitive abilities. Most 
of the activities are cognitively undemanding. It cannot be disregarded wholly 
because it boosts up learners’ confidence. There should be balance between LOTS 
(lower order thinking skills) and HOTS (higher order thinking skills) activities in 
the book. As at intermediate level students are given questions of analysis and 
synthesizing to enhance their cognitive abilities. No such projects are included as 
are the best source to evaluate higher order skills in the learners. The focus is 
only on accuracy by practicing grammar and controlled writing skills. 


3. 4 Communication 


IV. COMMUNICATION 

Yes 

No 

34. Does it provide support to simplify language? 


V 

35. Does its emphasis communicative competence activities? 



36. Are activities developed to encourage teacher-student and 
student-student communication? 


V 

37. Do the activities enable students to use the L2 outside the 

classroom situations? 


V 

38. Are activities balance between individual response, pair work 
and group work? 


V 


Communication is considered one of the primary goals of language and 

considered as core concept in foreign language learning. For this reason, CLIL is 
more focused on fluency rather than grammar (Coyle, 2007). In CLIL, the focus is on 
using language to learn rather learning to use language. In language subject, the 
main emphasis is on vocabulary acquisition and structure progression. In the selected 
textbook, there is no practice of simplified use of language in the form of dialogue 
and conversation that can present the realistic picture of the target culture of 
second/foreign language to the learners. As a result, communicative competence in 

English language cannot be achieved as a learning outcome. The textbook does not 
follow any communicative model of the language. The textbooks should be organized 
according to language model so that element of realism can be incorporated in 
materials as it stays learners motivated. But this textbook is based on GTM and no 

communicative approach is incorporated. In order to make textbooks close to 
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learners’ need and interest, there should be humanistic activities in language 
teaching materials. Humanistic activities engage learners in interesting way to learn 
the language. Learners of intermediate level are required to improve their 
communication skills so that they can perform best in their higher studies. But the 
textbook provides no opportunity for learner-learner interaction or learner-teacher 
interaction so that they can improve their communicative and speaking skills. The 
textbook does not include grammar, listening and creative writing activities at all. 
For this reason, students after passing their intermediate level are unable to 
perform outside the classroom. Moreover, oral skills activities (e. g. dialogues and 
discussion among different groups of students) intended to facilitate learning 
through genuine interaction and the language skills and sub-skills are presented and 
practiced. So, the textbook fails to develop communicative strategies among learners. 

3. 5 Culture 


V. CULTURE 

Yes 

No 

39. Does it relate content to the learners’ culture and environment? 


✓ 

40. Does it guide students in developing cultural awareness? 

V 


41. Do the content relevant to the socio-cultural environment? 



42. Does the content involve culture-specific items? 

V 


43. Is the content free from stereotypical images? 



44. Do the visuals relate to the students own cultural settings? 



45. Have cultural sensitivities been considered? 

V 



In English language textbooks, there is an interdependent relation between 
language and culture. Coyle (2007) considers that the rightful place of culture is 
central to CLIL. Cultural awareness is developed by in-depth understanding of own and 
others’ culture and a positive interest in both cultures i. e. how they connect and 
are similar to each other. 

In the selected textbook, the content does not represent learner’ s 
environment and culture. Similar findings have been observed in the result of Shah, 
Majeed, ffaheed and Anjum (2013) while analyzing the PTB textbook through a 
questionnaire. They observe that the textbook is boring and learners do not take much 
interest in it while learning through textbooks. The textbooks do not represent 
learners’ culture so they do not focus on it. According to the responses of the 
teachers, they also want to change foreign elements in the textbook with respect to 
the cultural elements. Though it provides cultural awareness about other culture yet 
the textbook included two stories from Pakistani culture so that the learners do not 
get bore or feel themselves as aliens in the foreign content. There is a 
stereotypical image of the foreign and local culture. There is no linguistic 
difficulty or cultural sensitivity in the content. As the material is selected by 
local bodies and is written by the natives because of that they are far away from the 
native culture (Shah et al., 2013). Kausar, Mushtaq and Badshah (2016) analysed the 
intermediate English textbook and found no cultural biasness. There is no use of such 
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visuals as depict cultural stereotypes. Prodromou (1988) and Alpetekin (1993) suggest 
that inclusion of foreign culture in ELT textbooks has a potential to create serious 
cultural misunderstandings or comprehension problems that might lack proper schemata 
to interpret the foreign content. On the whole, the book is free from cultural 
sensitivities but there should be balance between target and local culture to make 
the textbook user-friendly. 

4 Conclusion 

The aim of this research was to investigate CLIL approach in an intermediate 
level textbook through textbook evaluation. After the analysis through a CLIL 
checklist, it has been observed that there is no integration of CLIL principles i. e. 
content, culture, cognition and communication in the intermediate English textbook. 
There are many shortfalls in the textbook regarding its content organization, 
activities engage in critical thinking, lack of authentic material to achieve 
communicative competence, neglecting integrating language perspective. This indicates 
that material is not appropriate to ESL/EFL setting. One of the major shortfalls is 
that the practice of listening and speaking skills is neglected in the exercises and 
major focus is laid on questions based on reading of lesson content or grammar, (e. g. 
grammar, punctuation, choose right form of verb, synonyms). There is no such activity 
as fosters critical thinking in learners. The content presents objective view of 
culture and involves vocabulary to increase learners’ cognitive progression. 
Moreover, in CLIL context, language use is different from the language use in 
language classrooms i. e. the students need to argue, discuss, justify, explain and 
debate the issues linked with language content use and adequate content for specific 
vocabulary studies. In intermediate textbook there is no interaction between student- 
student interaction and student-teacher interaction. The textbook lacks in providing 
realistic situations of the target as well as learners’ own culture regarding 
everyday conversations that is the foremost need of the ESL/EFL learners. In 
addition, the textbook neglected the dimensions of language of learning and language 
for learning. Language through learning refers to the notion that language learning 
cannot take place without active participation in thinking and language. CLIL 
classroom settings demand different levels of interaction and talking as compared to 
traditional language classroom. The textbook uncovers GTM as teaching methodology and 
classroom is teacher-centered and opposes curriculum policy claim. Therefore, the 
textbook does not promote lower order and higher order skills in a balanced way. Most 
of the exercise questions cover only first two levels of Bloom’ s taxonomy (i. e. 
knowledge and comprehension). 
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